How Discover & Nationwide Build Trust in Data with Anomalo and Alation

Data quality management has long been a complex hurdle for businesses, especially those dealing with large volumes of data. Poor data quality can have far-reaching effects, impacting decision-making and overall business performance.

At the revAlation 2023 conference, we had the privilege of hearing from companies and participants who are leveraging Anomalo and Alation to address their data quality challenges. They emphasized the importance of implementing a robust data quality management system to proactively detect and root-cause data issues before they derail business operations. This underlines how the fusion and seamless integration of data quality monitoring and a data catalog, offered by Anomalo and Alation, are pivotal in ensuring data-driven success for organizations.

In this article, we’ll distill the key takeaways from the conference, including a panel conversation featuring participants from Discover, Nationwide, HarbourVest, and Wind Creek.

‍

Scale is about much more than data size

The panel unanimously agreed that scaling data quality management involves much more than handling large datasets. They advocated taking a holistic view of the user lifecycle: initial setup, ongoing management, efficient triage, smooth user experience, and ability to empower business users to resolve their own issues. At each of these touchpoints, Anomalo’s ability to operate at scale make it an effective data quality monitoring tool for modern enterprises.

Prakash Jaganathan, Senior Director of Enterprise Data Platforms from Discover, shared that Discover processes up to half a trillion dollars in annual transaction volume. Because of the sheer size and complexity of their datasets, they needed a comprehensive tooling solution that was rooted in machine learning because a solely manual approach (data stewards writing data validation rules) would not work. Prakash stressed that:

“The ease of setting up unsupervised machine learning monitoring with Anomalo was a differentiator.”

Prakash Jaganathan
Senior Director of Enterprise Data Platforms at Discover

At this scale of data, it was important that this tool was flexible (able to handle 10 columns or up to 1000s of columns) and able to send notification alerts to a variety of channels (Slack, Microsoft Teams, Incident Tracker Systems, etc). But beyond that, Discover sought a tool capable of providing alerts backed by rich, actionable data visualizations and explanations. As Prakash said, “What good is an anomaly alert if it takes 2 weeks to triage it?”

Steve Hotchkiss, Vice President of Data Management and Analytics at HarbourVest Partners, agreed with this sentiment. He explained that he wanted a blend of capabilities from his chosen data quality vendor. The tool needed an unsupervised machine learning algorithm approach that could detect anomalies and drift in data, as well as a low-code/no-code offering that could accommodate data stewards—people who have expert knowledge of data but don’t write SQL. These requirements were a key reason HarbourVest partnered with Anomalo.

Michael Randall (Director, Enterprise Data Governance at Nationwide) echoed the importance of the ML-based approach:

All data programs have checks that help us find known issues. What Anomalo brings is that it finds issues we’re not looking for.

Michael Randall
Director, Enterprise Data Governance at Nationwide

Meeting diverse industry needs

Data quality management is a universal problem across industries. During the panel conversation, we heard insights from participants who expressed how data quality management tools have improved their business operations.

In the property insurance sector, external factors like inflation and climate change make it harder to plan effectively. Michael Randall, Director of Enterprise Data Governance at Nationwide, highlighted how data quality tools are essential as they empower leaders to make better decisions in response to such dynamic conditions.

Such tools are crucial for managing internal operational issues as well. Michael shared how at Nationwide, a gap in claims data during a closing financial quarter led to a filing delay. He explained that a tool like Anomalo could have identified the issue immediately and saved them the stress.

Jagadeesh Tupakula, Director of Data Strategy and Analytics at Wind Creek Hospitality, described how his company migrated to the cloud two years ago. The process was complicated in many dimensions—data from different sources, different formats, and in different environments. Relying on manual data quality checks would’ve been highly time consuming and cumbersome. So the team looked for a vendor that could save them engineering pain by pinpointing data quality issues and sending actionable alerts before business analysts noticed any issues.

Wind Creek also chose Anomalo for its out-of-the-box integration with Alation. By combining Alation and Anomalo, they were able to identify and determine the root cause of data issues, whether they arose in the transformation or source system. They also gained a better understanding of their data—how it flowed through their systems and how it changed. These capabilities were major selling points for Wind Creek.

‍

Optimizing Data Management: The Alliance of Anomalo and Alation

The panelists shed light on the collaborative strength of using Anomalo with Alation. When Discover was looking for a data quality monitoring partner, they initially considered building one in-house. But Prakash mentioned that despite their strong engineering team, choosing the alliance of Anomalo and Alation offered faster time to market, and a lower total cost of ownership. Given the wide variety of their customer personas (data analysts, data stewards, data engineers, data governors etc), he stressed the need for a partnership approach because:

”No one vendor can solve all enterprise use cases.”

Prakash Jaganathan
Senior Director of Enterprise Data Platforms at Discover

Michael from Nationwide pointed out the synergy between Alation’s data discovery capabilities and Anomalo’s prowess in diagnosing data problems. This combination allowed their internal customers to find, understand, and trust their data. Jagadeesh emphasized this point: Alation provides a comprehensive view of all their business departments (hotel reservations, food & beverage, etc.) in a single view, while Anomalo allows Wind Creek Hospitality to profile and review data on a column by column basis.

Prakash (Discover) expressed that after working with Anomalo for two years, the collaboration has shown remarkable results. Discover is managing about 1000 tables with Anomalo, serving 35 internal teams, and plans to scale up by 100 tables per week. After conducting numerous product demos to different teams within the company Prakash noted the tangible enthusiasm. The interest was particularly evident among non-data engineers, who previously never had a tool to influence data quality monitoring. Once they saw what Anomalo could do for them, their reactions were immediate: “When can I get my hands on this?”

In summary, the Anomalo-Alation partnership marks a pivotal step in data quality management. By merging Anomalo’s advanced data monitoring with Alation’s data cataloging, this collaboration equips businesses to handle complex data challenges more efficiently and confidently. The union is more than a set of tools—it’s a holistic strategy that ensures reliable data, operational efficiency, and informed decision-making across sectors.

To understand the impact of this partnership, we encourage you to experience it firsthand. Sign up here for a demo and see how we can revolutionize your approach to data quality management.